AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers

To date, endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. Prolonged Fluoroscopic exposure is harmful for the patient and the clinician, and may lead to severe post-operative sequlae such as the development of cancer. Meanwhile, the use of interventional Ultrasound has gained popularity, due to its well-known benefits of small spatial footprint, fast data acquisition, and higher tissue contrast images. However, ultrasound images are hard to interpret, and it is difficult to localise vessels, catheters, and guidewires within them. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences. The network architecture was inspired by the Attention in Attention mechanism, temporal tracking networks, and introduced a novel 3D segmentation head that performs 3D deconvolution across time. In order to facilitate training of such deep learning networks, we introduce a new data synthesis pipeline that used physics-based catheter insertion simulations, along with a convolutional ray-casting ultrasound simulator to produce synthetic ultrasound images of endovascular interventions. The proposed method is validated on a hold-out validation dataset, thus demonstrated robustness to ultrasound noise and a wide range of scanning angles. It was also tested on data collected from silicon-based aorta phantoms, thus demonstrated its potential for translation from sim-to-real. This work represents a significant step towards safer and more efficient endovascular surgery using interventional ultrasound.Comment: This work has been submitted to the IEEE for possible publicatio

Caveats on the first-generation da Vinci Research Kit: latent technical constraints and essential calibrations

Telesurgical robotic systems provide a well established form of assistance in the operating theater, with evidence of growing uptake in recent years. Until now, the da Vinci surgical system (Intuitive Surgical Inc, Sunnyvale, California) has been the most widely adopted robot of this kind, with more than 6,700 systems in current clinical use worldwide [1]. To accelerate research on robotic-assisted surgery, the retired first-generation da Vinci robots have been redeployed for research use as "da Vinci Research Kits" (dVRKs), which have been distributed to research institutions around the world to support both training and research in the sector. In the past ten years, a great amount of research on the dVRK has been carried out across a vast range of research topics. During this extensive and distributed process, common technical issues have been identified that are buried deep within the dVRK research and development architecture, and were found to be common among dVRK user feedback, regardless of the breadth and disparity of research directions identified. This paper gathers and analyzes the most significant of these, with a focus on the technical constraints of the first-generation dVRK, which both existing and prospective users should be aware of before embarking onto dVRK-related research. The hope is that this review will aid users in identifying and addressing common limitations of the systems promptly, thus helping to accelerate progress in the field.Comment: 15 pages, 7 figure

Effect of tissue permeability and drug diffusion anisotropy on convection-enhanced delivery

Peer reviewedPublisher PD

Pose Measurement of Flexible Medical Instruments Using Fiber Bragg Gratings in Multi-Core Fiber

Accurate navigation of flexible medical instruments like catheters require the knowledge of its pose, that is its position and orientation. In this paper multi-core fibers inscribed with fiber Bragg gratings (FBG) are utilized as sensors to measure the pose of a multi-segment catheter. A reconstruction technique that provides the pose of such a fiber is presented. First, the measurement from the Bragg gratings are converted to strain then the curvature is deduced based on those strain calculations. Next, the curvature and the Bishop frame equations are used to reconstruct the fiber. This technique is validated through experiments where the mean error in position and orientation is observed to be less than 4.69 mm and 6.48 degrees, respectively. The main contributions of the paper are the use of Bishop frames in the reconstruction and the experimental validation of the acquired pose

Automatic optimized 3D path planner for steerable catheters with heuristic search and uncertainty tolerance

In this paper, an automatic planner for minimally invasive neurosurgery is presented. The solution can provide the surgeon with the best path to connect a user-defined entry point with a target in accordance with specific optimality criteria guaranteeing the clearance from obstacles which can be found along the insertion pathway. The method is integrated onto the EDEN2020∗ programmable bevel-tip needle, a multi-segment steerable probe intended to be used to perform drug delivery for glioblastomas treatment. A sample-based heuristic search inspired to the BIT* algorithm is used to define the optimal solution in terms of path length, followed by a smoothing phase required to meet the kinematic constraint of the catheter. To account for inaccuracies in catheter modeling, which could de- termine unexpected control errors over the insertion procedure, an uncertainty margin is defined so that to include a further level of safety for the planning algorithm. The feasibility of the proposed solution was demonstrated by testing the method in simulated neurosurgical scenarios with different degree of obstacles occupancy and against other sample-based algorithms present in literature: RRT, RRT* and an enhanced version of the RRT-Connect

SimPS-Net: Simultaneous Pose & Segmentation Network of Surgical Tools

Localisation of surgical tools during operation is of paramount importance in the context of robotic assisted surgery. 3D pose estimation can be utilised to explore the interaction of tools with registered tissue and improve the motion planning of robotic platforms, thus avoiding potential collisions with external agents. With the problems of traditional tracking systems being cost and the need to redesign surgical tools to accommodate markers, there has been a shift towards image-based, markerless tracking techniques. This study introduces a network capable of detecting and localising tools in 3D using a monocular setup. For training and validation, a novel dataset, 3dStool, was produced, and the network was trained to obtain a mean Dice coefficient of 85.0% for detection, along with a mean position and orientation error of 5.5mm and 3.3. respectively. The presented method is significantly more versatile than various state of the art solutions, as it requires no prior knowledge regarding the 3D structure of the tracked tools. The results were compared to standard pose estimation networks using the same dataset and demonstrated lower errors along most metrics. In addition, the generalisation capabilities of the proposed network were explored by performing inference on a previously unseen pair of scissors

CathSim: An Open-source Simulator for Autonomous Cannulation

Autonomous robots in endovascular operations have the potential to navigate circulatory systems safely and reliably while decreasing the susceptibility to human errors. However, there are numerous challenges involved with the process of training such robots such as long training duration due to sample inefficiency of machine learning algorithms and safety issues arising from the interaction between the catheter and the endovascular phantom. Physics simulators have been used in the context of endovascular procedures, but they are typically employed for staff training and generally do not conform to the autonomous cannulation goal. Furthermore, most current simulators are closed-source which hinders the collaborative development of safe and reliable autonomous systems. In this work, we introduce CathSim, an open-source simulation environment that accelerates the development of machine learning algorithms for autonomous endovascular navigation. We first simulate the high-fidelity catheter and aorta with the state-of-the-art endovascular robot. We then provide the capability of real-time force sensing between the catheter and the aorta in the simulation environment. We validate our simulator by conducting two different catheterisation tasks within two primary arteries using two popular reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). The experimental results show that using our open-source simulator, we can successfully train the reinforcement learning agents to perform different autonomous cannulation tasks

Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation

Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 2.55 arcmin for VST and 7.72 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 1.04 mm, 5.96 2.22 for VST system and 4.36 0.80 mm, 5.65 1.42 for OST system